Mixed Optimization for Smooth Functions
نویسندگان
چکیده
It is well known that the optimal convergence rate for stochastic optimization of smooth functions is O(1/ √ T ), which is same as stochastic optimization of Lipschitz continuous convex functions. This is in contrast to optimizing smooth functions using full gradients, which yields a convergence rate of O(1/T ). In this work, we consider a new setup for optimizing smooth functions, termed asMixed Optimization, which allows to access both a stochastic oracle and a full gradient oracle. Our goal is to significantly improve the convergence rate of stochastic optimization of smooth functions by having an additional small number of accesses to the full gradient oracle. We show that, with an O(lnT ) calls to the full gradient oracle and anO(T ) calls to the stochastic oracle, the proposed mixed optimization algorithm is able to achieve an optimization error of O(1/T ).
منابع مشابه
Quasi-Gap and Gap Functions for Non-Smooth Multi-Objective Semi-Infinite Optimization Problems
In this paper, we introduce and study some new single-valued gap functions for non-differentiable semi-infinite multiobjective optimization problems with locally Lipschitz data. Since one of the fundamental properties of gap function for optimization problems is its abilities in characterizing the solutions of the problem in question, then the essential properties of the newly introduced ...
متن کاملMixedGrad: An O(1/T) Convergence Rate Algorithm for Stochastic Smooth Optimization
It is well known that the optimal convergence rate for stochastic optimization of smooth functions is O(1/ √ T ), which is same as stochastic optimization of Lipschitz continuous convex functions. This is in contrast to optimizing smooth functions using full gradients, which yields a convergence rate of O(1/T ). In this work, we consider a new setup for optimizing smooth functions, termed as Mi...
متن کاملA Multi-Objective Particle Swarm Optimization for Mixed-Model Assembly Line Balancing with Different Skilled Workers
This paper presents a multi-objective Particle Swarm Optimization (PSO) algorithm for worker assignment and mixed-model assembly line balancing problem when task times depend on the worker’s skill level. The objectives of this model are minimization of the number of stations (equivalent to the maximization of the weighted line efficiency), minimization of the weighted smoothness index and minim...
متن کاملPERFORMANCE OF DIFFERENT ANT-BASED ALGORITHMS FOR OPTIMIZATION OF MIXED VARIABLE DOMAIN IN CIVIL ENGINEERING DESIGNS
Ant colony optimization algorithms (ACOs) have been basically introduced to discrete variable problems and applied to different research domains in several engineering fields. Meanwhile, abundant studies have been already involved to adapt different ant models to continuous search spaces. Assessments indicate competitive performance of ACOs on discrete or continuous domains. Therefore, as poten...
متن کاملOn Second-Order Necessary Conditions for Broken Extremals
We consider optimal control problems with initial–final state equality and inequality constraints and running mixed state-control equality constraints given by smooth functions. The mixed constraints satisfy the regularity assumption of linear independence of gradients with respect to the control. We present simple proofs of second-order necessary conditions for (extended) weak minimum for extr...
متن کامل